1. Field of the Invention
This invention relates to the field of video processing, and in particular to the decoding of enhanced video in a select window within an MPEG base-layer window.
2. Description of Related Art
MPEG encoding of video images supports the use of “Scalable Extensions” to a base-layer video stream. A scalable video stream is configured to allow decoders of different complexity to process the stream, to produce decoded images of different quality. The MPEG scalable extensions include, for example, spatial, temporal, SNR, data, and others.
A spatial extension to an MPEG encoding adds a higher-resolution encoding of the images to an encoding of a lower-resolution base-layer. If a user's processing power is limited, the user may choose to decode only the base-layer, to assure continuous streaming at the encoded frame rate. Alternatively, the user may choose to also decode the enhancement-layer to obtain higher resolution images, albeit with the potential for ‘freeze’ frames, or other visual anomalies, if the decoding process is not able to keep up with the encoded frame rate.
A temporal extension to an MPEG encoding adds additional frames to the base-layer frames. The base-layer frames are encoded at a somewhat low frame rate, and the additional frames increase the frame rate. Depending upon the processing power available, a user may choose to view either the low frame rate sequences of images, or the high frame rate sequences. Again, however, if the processing power is insufficient to support the decoding of the images at the higher frame rate, visual anomalies will occur.
In like manner, the SNR extension adds material to the base-layer images to improve the signal-to-noise ratio of the displayed images, and the data extension allows less-critical data, such as high-frequency components of the image, to be omitted from a base-layer encoding, and supplied by the optionally selectable data extension.
In each of the above examples, the user is generally provided an “all or nothing” choice. Either the spatial enhancement decoding is on, or it's off; additional frames are added, or they're not; and so on. A conventional method of allowing for intermediate choices is to provide different levels of each enhancement. For example, a medium-resolution enhancement-layer may be provided, and the high-resolution enhancement-layer is built upon the medium-resolution enhancement-layer. In this manner, the user can select between low, medium, and high resolution, depending upon the available processing power, and depending upon the user's particular preference for resolution. In this case, however, the user is still provided with an “all or nothing” choice, except with varying definitions of “all”. That is, if the user selects “medium” resolution, all frames will be decoded at the medium resolution. If the decoding processor cannot keep up with the decoding of the medium resolution, visual anomalies occur.